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TO  ERROR  CONTROL 
J.  M.  Yohe 

1.  Introduction. 

Interval  analysis,  far  from  being  just  an  academician's  toy,  provides 
a viable  means  for  uncovering  sources  of  potential  error  in  digital  computations. 
In  this  note,  we  examine  some  of  the  ways  in  which  interval  analysis  can  be 
employed  to  enhance  the  accuracy  of  computations,  whether  or  not  it  is  em- 
ployed in  the  production  environment;  we  also  suggest  guidelines  which  can 
help  determine  whether  interval  analysis  should  be  used  in  the  production  situation. 

The  basic  reference  for  interval  analysis  is  the  book  by  R.  E.  Moore 
[3].  For  the  sake  of  completeness,  however,  we  will  sketch  the  theory  very 
briefly. 

A (real)  interval  [a,  b]  is  defined  in  the  usual  fashion: 

[a,b]  = ( x | a <x  <b}  . 

A real-valued  function  f of  a real  variable  can  be  extended  to  an  interval- 
valued function  of  an  interval  variable  by  defining 
f([a,  b])  = (f(x)  | x € [a,  b]}  , 

assuming,  of  course,  that  f is  defined  and  continuous  on  [a,  b].  Real- 
valued functions  of  several  real  variables  are  similarly  extendable,  and  we 
can  thus  define  the  usual  arithmetic  operations  on  intervals  (except,  of  course, 
for  division  by  an  interval  containing  zero). 

Intervals  can  be  regarded  as  representing  real  numbers  where  some 

uncertainty  is  present.  Degenerate  intervals  (intervals  whose  left  endpoint 
Sponsored  by  the  United  States  Army  under  Contract  No.  DAAGZ9-75-C-0024 
and  a grant  from  the  Waterways  Experiment  Station. 


is  the  same  as  the  right)  would  then  represent  exact  real  numbers;  indeed, 
interval  operations  and  functions  on  degenerate  intervals  are  the  same  as 
their  real  counterparts.  We  thus  identify  the  real  numbers  with  degenerate 
intervals,  and  we  note  that  the  interval  number  system  is  a bona  fide  exten- 
sion of  the  reals. 

In  computing,  of  course,  the  ideal  is  seldom  attainable.  Even  when 
we  begin  by  using  exact  data,  uncertainty  creeps  in  due  to  roundoff  error. 

Since  the  result  of  applying  a given  function  to  machine  representable  argu- 
ments is  often  not  machine  representable,  it  rapidly  becomes  necessary  to 
work  with  approximations.  Depending  on  several  factors,  including  hardware 
behavior,  algorithm  stability,  and  data,  the  approximation  will  be  more  or 
less  accurate. 

In  interval  analysis,  roundoff  error  is  handled  by  making  certain  that 
the  left  endpoint  of  the  interval  is  always  rounded  down,  and  the  right  endpoint 
is  always  rounded  up.  A formal  treatment  of  interval  analysis  on  finite-preci- 
sion number  spaces  may  be  found  in  Kulisch  [1]. 

In  the  case  of  ordinary  computer  arithmetic,  roundoff  error  may  accumu- 
late to  the  point  where  the  result  of  the  computation  has  no  significance.  In 
interval  arithmetic,  however,  accumulated  roundoff  error  will  cause  the  inter- 
vals to  become  wide,  but  the  interval  will  always  contain  the  true  result. 

Thus  the  user  can  get  some  idea  of  how  stable  the  algorithm  is  with  the  given 

* | 

data,  and  how  uncertain  the  answers  may  be. 

ft 

In  the  following  sections,  we  give  some  examples  of  ways  in  which 

t 

interval  analysis  can  reveal  instabilities  in  algorithms  or  weaknesses  in 
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interval  aritnmetic  as  implemented  in  the  INTERVAL  II  package  of  the 
Mathematics  Research  Center.  Results  are  given  in  octal,  since  the  1110  is 
a binary  machine;  however,  since  the  octal  number  system  is  foreign  to  most 
of  us,  approximate  decimal  values  are  given  for  reference. 

2.  Summation 

In  elementary  numerical  analysis,  one  learns  that  the  order  in  which 

terms  of  a summation  are  added  can  affect  the  accuracy  of  the  result.  To 

illustrate  this,  we  considered  the  alternating  sum 

128 

y = Z (-1/S)1  . 

i=0 

We  summed  this  series  in  both  directions:  i.e.  , with  the  largest  term 
first  (from  the  left)  and  with  the  smallest  term  first  (from  the  right).  The  re- 
sults are  shown  in  Table  1.  Note  that  the  sum  of  the  corresponding  infinite 
series  is  5/6,  and  that  the  correctly  rounded  9 -digit  octal  fraction  corres- 
ponding to  5/6  is  .652525253.  While  the  1110  floating  point  arithmetic  pro- 
duced results  which  look  reasonably  good,  note  that  the  better  of  the  two 
differs  from  the  correctly  rounded  value  by  one  bit  in  the  low-order  position. 
(The  fact  that  it  does  not  even  lie  in  the  interval  computed  by  the  Interval  II 
package  is  due  to  a hardware  anomaly:  see  Section  4 for  details.  ) 

The  interval  arithmetic  results  are  more  revealing.  When  we  sum  the 
series  from  the  right,  we  obtain  bounds  that  are  best  possible;  i.e.  , the 
upper  and  lower  bounds  differ  by  just  one  bit  in  the  least  significant  position. 
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TABLE  1 
128 

Summation  of  ^ ( - 1/ 5 ) 

i^O 


i 

\ 


Summation 
beginning 
with  l = 

Interval 

1110  floating 
point 

0 

octal 

(.  6 52  525  154, 

.652  525  354) 

. 652  525  255 

decimal 

(.83333  28644, 

.83333  38201) 

.83333  33507 

128 

octal 

(.652  525  252, 

.652  525  253) 

. 652  525  254 

decimal 

(.83333  33284, 

.83333  33358) 

.83333  33433 

Summing  from  the  left,  however,  the  upper  and  lower  bounds  differ  by  a 
whopping  200  | In  the  light  of  the  fact  that  the  1110  produced  a much 
closer  answer  using  standard  floating  point  arithmetic,  this  might  seem  to 
call  interval  arithmetic  into  question.  However,  it  would  not  have  been  dif- 

m 

ficult  to  produce  a collection  of  terms  whose  true  sum  would  have  been  very 
close  to  one  extreme  or  the  other,  and  yet  would  have  given  the  same  result 
as  the  above  numbers  gave  on  the  1110.  Since  the  numbers  involved  in  a real- 
life  computation  are  seldom  as  clean  as  the  ones  in  our  example,  the  width 
of  the  interval  turns  out  to  be  a strength  of  interval  arithmetic  rather  than  a 
weakness;  it  clearly  reveals  the  advantage  to  be  gained  by  careful  attention 
to  the  order  of  summation.  In  many  cases,  the  relative  sizes  of  the  terms  of 
a summation  will  not  be  known  in  advance  (as  they  were  here);  in  such  cases, 
interval  arithmetic  can  give  an  excellent  idea  of  the  possible  roundoff  error 
involved  in  the  summation. 
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One  of  the  simplest  examples  of  how  seriously  roundoff  error  can 


affect  the  results  of  a computation  is  the  classic  example  of  solving  for  the 

roots  of  the  equation  ax^+bx+c  = 0.  In  the  case  where  4ac  is  small  compared  to 

2.  2 

b , so  that  the  discriminant  is  very  nearly  b , the  relative  error  in  the  root 

with  smaller  absolute  value  can  be  very  high  indeed  if  the  root  is  calculated 
by  blind  application  of  the  quadratic  formula.  This  is  not  a new  fact;  indeed, 
it  is  discussed  in  many  elementary  numerical  analysis  texts.  The  fix  is  sim- 
ple; one  calculates  the  root  with  larger  absolute  value  (the  one  using  addi- 
tion in  the  numerator  of  the  quadratic  formula  if  b is  negative,  or  subtrac- 
tion if  b is  positive)  using  the  quadratic  formula,  then  finds  the  other  root 
by  dividing  the  known  root  into  c/a. 

Naturally,  the  errors  involved  are  the  most  dramatic  when  the  coef- 
ficients of  the  quadratic  equation  are  close  to  the  limits  of  the  hardware's 
ability  to  produce  results  with  a modicum  of  accuracy.  For  purposes  of  illus- 
tration, we  took 


b = -1 
c = i. 

We  solved  for  both  roots  using  the  quadratic  formula,  and  then  solved  for  the 
root  with  smaller  absolute  value  using  the  above  algorithm.  The  results  are 
shown  in  Table  2. 

Several  observations  are  in  order  here.  The  floating  point  arithmetic 
on  the  1110  gives  a result  of  zero  for  the  second  root  using  the  quadratic 
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TABLE  2 

Roots  of  a quadratic  equation 

Root 

Interval 

1110  floating  point 

Larger 

oct 

(3  777  777  774,  4 000  000  000) 

4 000  000  000 

dec 

( 5368  70908,  5368  70912) 

5368  70912 

Smaller, 

by 

quadratic 

formula 

oct 

(0.000  000  000,  2.000  000  000) 

0. 000  000  000 

dec 

( 0,  2) 

0 

Smaller, 

by 

division 

oct 

(1.000  000  000,  1.000  000  002) 

1.  000  000  000 

dec 

(1.00000  00000,  1.00000  0015  ) 

1. 00000  00000 

formula.  It  is  quite  easy  to  check  that  this  is  nonsense;  substituting  this 
back  into  the  original  equation  gives  the  equation  1=0.  But  in  many  com- 
putations, the  result  of  one  calculation  is  simply  used  in  a further  calcula- 
tion without  being  checked;  the  error  is  not  discovered  until  later,  if  at  all. 
The  interval  analysis  result  is  the  interval  (0,2),  which  is  almost  uselessly 
wide  --  but  the  width  of  the  interval  points  out  the  instability  of  the  algorithm 
for  the  given  data. 

Passing  to  the  more  accurate  results,  note  that  here  also  the  interval 

result  gives  more  information.  The  standard  arithmetic  states  flatly  that  the 
29 

larger  root  is  2 ; but  again,  substituting  this  in  the  original  equation  gives 

the  result  1 = 0.  To  an  unsophisticated  program,  this  root  might  appear  to 
be  just  as  questionable  as  the  value  of  zero  obtained  by  the  quadratic  formula 
for  the  other  root.  But  the  interval  analysis  package  tells  us  that  the  true 
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29  29 

root  lies  somewhere  between  2 -4  and  2 ; these  are  the  closest  bounds 

available  on  the  1110.  That  is,  the  calculation  has  been  performed  with  the 
1110' s maximum  accuracy.  Likewise,  the  smaller  root  is  shown  to  be  calcul- 
ated with  maximum  accuracy;  and  indeed,  the  interval  package  lets  us  deduce 
that  the  root  is  greater  than  or  equal  to  one,  if  we  are  interested  in  knowing 
that.  The  ordinary  floating  point  arithmetic  leaves  us  to  guess  which  side 
of  1 the  root  lies  on. 

4.  Hardware  Limitations: 

Unfortunately,  present-day  hardware  is  not  always  as  well -designed 
as  it  could  be;  proper  rounding  is  often  not  available.  This  can  result  in  un- 
warranted error  accumulation.  The  following  example,  which  we  ran  on  the 
UNIVAC  1110,  illustrates  this  rather  dramatically. 

We  computed  X(l)  :=  1.  0 - 2~27  - 2~ 27 . On  the  1110,  this  is  not  the 
“ 2 

same  as  1.0  - 2 , since  the  1110  carries  a 27 -bit  fraction  in  its  floating 

point  representation,  and  truncates  after  equalizing  exponents  but  before  ex- 
ecuting the  add  or  subtract  operation.  The  result  of  this  is  that,  although 
1.0  - 2 2/  is  representable  on  the  1110,  it  can  not  be  computed  by  performing 
the  indicated  subtraction. 

2 

Having  computed  X(l),  we  computed  X(I)  :=  X(I-l)  for  I :=  2,3,..., 

40.  Finally,  we  summed  the  X(I)'s  from  both  left  and  right.  The  results, 
both  in  interval  arithmetic  and  in  standard  1110  arithmetic  are  shown  in  Table 
2.  We  have  shown  the  octal  results  in  standard  1110  floating  point  format; 
the  first  three  octal  digits  are  the  exponent  part  of  the  number  and  the  last 
nine  are  the  fractional  part.  The  exponent  is  the  appropriate  power  of  2 , 


l 
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Intarval 


#000000000t02  205652531051  205652531060  206580000000 


biased  by  200  . 

O 

The  table  is  self-explanatory.  However,  we  can  not  help  noting 
that,  even  in  the  worst  case,  the  interval  arithmetic  package  gives  answers 
that  are  far  better  than  the  1110's  floating  point  arithmetic  yields;  the  true 
answer  has  to  lie  somewhere  between  25.6  and  26.7:  yet  the  ill 0 computes 
401 

The  1110  is  not  unique  in  its  foibles;  many  other  computers  have 
anomalies  in  their  arithmetic  operations.  We  even  know  of  one  series  of 
computers  in  which  1.0  is  not  a multiplicative  identity,  owing  to  trunca- 
tion prior  to  post-normalization'. 

5.  Analysis  of  algorithm  limitations: 

A good  background  in  numerical  analysis  will  enable  one  to  avoid 
many  of  the  obvious  sources  of  numerical  error,  such  as  the  ones  we  have 
displayed  in  Sections  2 and  3.  However,  even  the  best  of  algorithms  may 
be  somewhat  data  sensitive,  and  other  algorithms  in  common  use  may  not 
be  the  best  that  can  be  devised. 

The  reliability  of  a given  algorithm  applied  to  a reasonable  data  set 
may  be  approximated  through  the  use  of  interval  analysis.  The  interval 
version  of  the  algorithm  should  be  run  using  several  representative  data 
sets,  and  the  widths  of  the  resulting  intervals  examined.  If  the  algorithm 
produces  acceptably  narrow  intervals  for  several  representative  data  sets, 
it  is  likely  that  the  algorithm  is  stable.  Interval  results  which  are  unaccept- 
ably wide  do  not  necessarily  indicate  a bad  algorithm,  but  simply  alert  one 
to  the  possibility  that  data  sensitivity  may  exist. 
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One  of  the  most  common  sources  of  pessimistic  interval  bounds  is 
the  repeated  use  of  the  same  variable  in  a mathematical  formula.  As  an 
extreme  example,  consider  y = x/x.  Clearly  the  result  should  be  exactly  1, 
and  would  be  in  most  if  not  all  floating  point  arithmetic  systems.  But  if  x 
is  the  interval  ( |,3),  then  y would  be  the  interval  (1/6,6)  - - a far  cry  from 
(1,  n:  The  trouble  is,  of  course,  that  the  interval  arithmetic  routines  do 

not  recognize  the  dependence. 

Nevertheless,  if  a computation  produces  intervals  of  unacceptable 
width,  it  is  a signal  that  the  computation  should  be  scrutinized  for  unstable 
operations.  It  may  be  necessary  to  insert  several  checkpoints  to  ascertain 
where  the  computation  loses  accuracy.  Once  the  trouble  spot  is  pinpointed, 
there  are  several  things  that  one  may  consider  in  attempting  to  solve  the 
problem : 

1.  Is  there  an  equivalent,  but  more  stable,  algorithm  which  can  be 
used?  The  square  root  example  illustrates  this  possibility. 

2.  Is  there  a critical  summation  which  could  be  made  more  accurate 
by  judicious  choice  of  the  order  of  summation?  It  might  even  be  worth  invok- 
ing a rather  sophisticated  sort-and-sum  algorithm  to  perform  the  summation. 

3.  Can  the  critical  portion  be  rewritten  using  higher  precision  arith- 
metic in  such  a way  as  to  improve  accuracy? 

The  latter  question  may  well  point  the  way  to  research  problems.  Not 
enough  attention  has  been  paid  to  the  problem  of  what  is  required  within  an 
algorithm  in  order  to  produce  a result  of  a given  precision.  Kulisch  [2]  has 
stated  that  inner  products  can  be  formed  to  full  single  precision  accuracy 
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using  an  accumulator  that  is  only  slightly  longer  than  double  precision; 
however,  for  many  types  of  problems  results  of  this  kind  are  unknown. 

6 . Analysis  of  system  limitations: 

The  example  of  Section  4 points  out  what  should  have  been  obvious 
from  the  beginning:  if  interval  analysis  is  to  be  used  for  the  purpose  of 
analyzing  a system  (hardware  and/or  software),  then  its  performance  must 
be  at  least  as  bad  as  the  performance  of  the  system  under  study.  INTERVAL  II 
was  designed  as  a production  package,  and  wherever  feasible  its  operations 
yield  results  of  the  greatest  possible  accuracy.  Thus,  in  its  present  form, 
it  can  not  be  used  effectively  to  predict  the  effects  of  hardware  and  software 
anomalies  on  the  results  of  a computation. 

If  this  sort  of  analysis  is  required,  the  INTERVAL  II  package  can  be 
modified  to  handle  it.  One  would  simply  use  as  a basis  the  data  type  under 
study,  supply  error  bound  information  on  the  mathematical  function  routines, 
and  rewrite  the  five  arithmetic  primitives  to  emulate  the  action  of  the  hard- 
ware and  at  the  same  time  obtain  rigorous  bounds  on  the  results  thus  obtained. 
In  particular,  the  upper  bound  for  an  operation  would  need  to  be  the  greater 
of  the  upper  bound  obtained  by  rounding  the  true  result  up  and  the  result 
actually  obtained  by  the  system  under  study;  the  lower  bound  would  be  treated 
analogously. 

Assuming  that  the  INTERVAL  II  package  has  been  thus  modified,  appro- 
priate test  programs  should  be  run  using  the  modified  INTERVAL  II,  and  the 
results  compared  with  the  results  obtained  when  the  unmodified  INTERVAL  II 
package  was  used.  The  difference  is  the  amount  by  which  system  inaccuracies 
could  affect  the  accuracy  of  the  results. 
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7 . Interval  analysis  as  a production  tool: 

In  cases  where  accuracy  is  of  critical  importance  and  instability  of 
the  algorithm  in  certain  situations  can  not  be  eliminated,  it  may  be  necessary 
to  verify  some  or  all  of  the  computation  by  using  interval  arithmetic.  At  pres- 
ent, there  is  indeed  a penalty  associated  with  doing  this;  the  arithmetic 
operations  are  likely  to  take  an  order  of  magnitude  or  two  more  time  in  inter- 
val arithmetic  than  in  standard  floating  point.  This  would  not  need  to  be  the 
case  if  hardware  were  designed  to  be  more  hospitable  to  interval  arithmetic  ; 
details  are  given  in  [4].  Surprisingly,  the  standard  mathematical  functions 
in  interval  arithmetic  are  not  that  much  more  expensive;  here  the  factor  is  two 
to  three  times  as  long.  Even  though  it  is  more  costly  in  terms  of  machine 
time,  it  may  be  worth  it:  technology  is  driving  hardware  prices  down  to  the 
level  where  inefficiency  is  no  longer  regarded  as  an  unpardonable  sin,  and 
the  extra  cost  may  seem  small  when  balanced  against  possible  failure  of  a 
structure  designed  using  the  results  of  computation. 

8 . Conclusions : 

We  have  given  simple  examples  of  the  ways  in  which  computer  algo- 
rithms can  be  data  sensitive  and  the  effects  that  roundoff  and  truncation  error 
can  have  on  computations.  We  have  also  pointed  out  how  a seemingly  in- 
significant design  lapse  in  one  production  computer  can  lead  to  ridiculous 
results  in  an  admittedly  contrived,  but  still  simple,  problem.  Finally,  we 
have  given  a few  elementary  guidelines  for  the  application  of  interval  arith- 
metic to  the  detection  and  control  of  error  in  both  hardware  and  software. 
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Certainly,  the  error  encountered  in  a computation  will  seldom  be  as 
great  as  might  be  indicated  by  the  results  of  performing  the  computation  in 
interval  arithmetic  --  but  it  might  be.  It  could  be  said  that  a given  bridge 
will  seldom  collapse,  for  having  once  done  so  it  no  longer  exists  and  there- 
fore cannot  collapse  again.  But  once  is  once  too  often,  both  in  terms  of 
bridges  collapsing  and  of  computers  producing  trusted  answers  that  are  not 
even  in  the  right  ball  park.  And  if  such  simple  problems  as  we  have  presented 
can  lead  to  such  dramatically  wrong  answers,  the  mind  boggles  at  what  could 
happen  in  the  sorts  of  complicated  calculations  that  we  routinely  use  in 
making  critical  decisions  today. 
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